I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Pull request type

Please check the type of change your PR introduces:

• Bugfix
• Feature
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Documentation content changes
• Other (please describe):

What is the current behavior?

Storage/Database CR creates one StatefulSet k8s object which operated pods with YDB processes.

Issue Number: YDBOPS-8645, YDBOPS-8646

What is the new behavior?

• We propose new CRDs (Custom Resource Definitions) named StorageNodeSet and DatabaseNodeSet, which define the parameters of StatefulSets for a specific logical group of YDB nodes. The creation and modification of objects occur through the definition in the Storage/Database object, specifying parameters in the spec.nodeSet field. Thus controller used in the ydb-operator performs the CRUD operation for *NodeSet CustomResource objects automatically.

• Performing Reconcile for NodeSet objects should include checking the presence of the StatefulSet resource, matching the desired quantity of replicas with the observed quantity. The status of parent objects when using *NodeSet is determined as the composition of readiness statuses StatefulSet and child Pod objects.

• Operations related to interaction with the YDB cluster, such as Storage initialization, Tenant creation, SelfCheck verification, are performed in the Reconcile process for the Storage/Database object. For this purpose, in the parameters of StatefulSet objects created using NodeSet, the spec.serviceName (within one parent object) must be specified identically.

Other information

Additional refactoring:

• Update Makefile test scenarios to use existing kind cluster

• Use new const variables GRPCServiceFQDNFormat and InterconnectServiceFQDNFormat to build URL endpoint and domain strings

• Edit const variable DefaultDatabaseDomain value from "root" to "Root"

• Remove database_defaults.go as useless (same logic as in kubebuilder webhook.Defaulter interface)

• Use type NamespacedRef for StorageClusterRef and DatabaseRef fields in CustomResources

• Add new field StorageEndpoint to Database and DatabaseNodeSet objects spec to define external FQDN for --node-broker container arg

• Remove useless PublicHost field in Database CR (use .spec.Service.GRPC.ExternalHost and .spec.Service.Interconnect.ExternalHost instead)

• Move YDB configuration build logic to a webhook so that it can be generated and specified when ydb-operator mutating API objects